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Key findings 



Since 2014, Florida law has required the 300 elementary schools with the 
lowest reading performance to provide supplemental reading instruction 
through an extended school day. This study found that in 2014/15, 
on average, the lowest performing schools were smaller than other 
elementary schools and served higher proportions of racial/ethnic minority 
students and students eligible for the federal school lunch program. 
Schools reported using a variety of strategies to comply with the extended 
school day policy such as increasing reading instruction time each day, 
increasing staff, providing professional development for teachers, and 
providing instruction in the extra hour that differed from instruction during 
the rest of the day. Increased professional development and curricular and 
pedagogic changes were identified as indirect benefits of implementation. 
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Summary 


Since the 2012/13 school year Florida law has required the 100 lowest performing elemen- 
tary schools in reading to extend the school day by one hour to provide supplemental 
reading instruction. In 2014 the law was broadened to include the 300 elementary schools 
with the lowest reading performance. 

A previous study of the state’s first two cohorts of 100 lowest performing schools found that 
observed growth in school reading performance after one to two years of implementing 
the extended school day policy did not exceed what would have been expected because 
of natural variation (Folsom et al., 2016). The current study follows up on that study by 
describing the location, demographic characteristics, and school reading performance 
of the 300 lowest performing schools and analyzing how the lowest performing schools 
implemented the extended school day policy (for example, the methods used to add the 
extra hour, staffing, and delivery of instruction). As with Folsom et al. (2016), the current 
study was based on a request from the Regional Educational Laboratory Southeast Improv- 
ing Literacy Alliance and Improving Low-Performing Schools Alliance — both of which 
have Florida Department of Education administrators as members. 

Based on publicly available data and survey and interview data collected by the Florida 
Department of Education, the study found: 

• The lowest performing elementary schools were located in 31 of Florida’s 65 school 
districts and in all five regions of the state. The lowest performing elementary 
schools were smaller on average than other elementary schools and enrolled a 
larger proportion of racial/ethnic minority students and students eligible for the 
federal school lunch program. Ninety-nine percent of the lowest performing ele- 
mentary schools were Title I schools. 

• On average, the school reading performance of the lowest performing elementary 
schools was 1.8 standard deviations below that of other elementary schools at the 
end of 2013/14 (the year they were identified as lowest performing) and 1.5 stan- 
dard deviations below that of other elementary schools at the end of 2014/15 (after 
implementing the extended school day policy for one or more years). 

• The elementary schools that implemented the extended school day policy report- 
ed using a variety of strategies such as increasing reading instruction time each 
day, increasing staff, providing professional development for teachers, and provid- 
ing instruction in the extra hour that differed from instruction during the rest of 
the day. 

• Participants identified indirect benefits of implementation, including perceived 
student gains. Interviewees attributed these gains not to the extra hour of instruc- 
tion but to improvements that occurred in conjunction with it, such as profes- 
sional development and curricular and pedagogic changes. However, the empirical 
analyses could not demonstrate whether student gains actually occurred. Barriers 
to implementation identified by interviewees included lack of resources and issues 
around logistical planning, such as short timelines. 
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Why this study? 


Since the 2012/13 school year Florida law has required the 100 lowest performing elemen- 
tary schools in reading to extend the school day by one hour (see box 1 for definitions of 
key terms used in the report). The extra hour must be spent on intensive, research-based 
reading instruction. 

A previous study of the state’s first two cohorts of 100 lowest performing schools found that 
observed growth in school reading performance after one to two years of implementing 
the extended school day policy did not exceed what would have been expected because 
of natural variation (Folsom et al., 2016). That study also investigated survey responses 
on how the 100 lowest performing schools implemented the extended school day policy 
and found that the schools reported increasing the amount of reading instruction time, 
increasing staff, and providing instruction in the extra hour that differed from instruction 
during the rest of the day. 

In 2014 the law was broadened to include the 300 lowest performing elementary schools. 
The increase in the number of schools required to implement the policy was accompanied 
by other modifications to the law. The law originally stated that the “additional hour of 
instruction must be provided only by teachers or reading specialists who are effective in 
teaching reading.” The modification added that the instruction could be provided “by a 
K— 5 mentoring reading program that is supervised by a teacher who is effective at teaching 
reading.” 1 

The Regional Educational Laboratory (REL) Southeast Improving Literacy Alliance and 
Improving Low-Performing Schools Alliance — both of which have Florida Department of 
Education administrators as members — requested an updated analysis of the 300 schools 
identified as the state’s lowest performing schools in 2014/15. Although the Florida Depart- 
ment of Education — through the Just Read, Florida! office and the Bureau of School 
Improvement — was responsible for overseeing implementation of the extended school day 
policy statewide, district- or school-level implementation was managed locally. Specifical- 
ly, the alliances were interested in the location, demographic characteristics, and school 
reading performance of the 300 lowest performing schools and in how the lowest perform- 
ing schools implemented the extended school day policy (for example, methods used to 
add the extra hour, staffing, and delivery of instruction), particularly given the changes in 
the language of the law. 2 While Folsom et al. (2016) used survey data to examine imple- 
mentation, the present study uses additional qualitative analysis of survey and interview 
data. 

The study was designed to describe how schools implemented the extended school day 
policy, not to determine whether the policy was effective. Therefore, this report does not 
offer recommendations on the best ways to incorporate the extra hour or on which aspects 
of policy implementation were effective. 

This study addressed four research questions: 

• Where were the lowest performing schools located, and how did their demograph- 
ic characteristics compare with those of other elementary schools? 

• How did the 2013/14 and 2014/15 school reading performance of the lowest per- 
forming schools compare with that of other elementary schools? 


This study updates 
a 2016 study of 
the state’s first 
two cohorts of 100 
lowest performing 
schools after 
Florida law was 
broadened to 
require the 300 
lowest performing 
elementary schools 
in reading to 
extend the school 
day by one hour 
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Box 1. Key terms 


Extended school day policy. The implementation of the law requiring the 300 lowest perform- 
ing elementary schools in reading to extend the school day by an hour. The extra hour must be 
spent on reading instruction provided by or supervised by teachers or reading specialists who 
are effective at teaching reading. 

Lowest performing schools. The 304 lowest performing schools in reading identified in 
2014/15 that were required to implement the extended school day policy. 1 These schools fell 
into three cohorts: 

• The lowest performing schools in 2014/15 (the 189 schools identified as a lowest per- 
forming school for the first time in 2014/15). 

• The lowest performing schools in 2014/15 and one prior year (the 93 schools identified 
as one of the 100 lowest performing schools in either 2012/13 or 2013/14 and as one of 
the 304 lowest performing schools in 2014/15). 

• The lowest performing schools in 2012/13-2014/15 (the 22 schools identified as one 
of the 100 lowest performing schools in 2012/13 and 2013/14 and as one of the 304 
lowest performing schools in 2014/15). 

Other elementary schools. The 1,494 elementary schools that were not identified as a lowest 
performing school in reading in 2014/15. These schools were not required to implement the 
extended school day policy in 2014/15. 

School reading performance. Between the 2013/14 and 2014/15 school years, Florida 
replaced the Florida Comprehensive Assessment Test 2.0 (FCAT 2.0) with the Florida Stan- 
dards Assessment (FSA). This report thus defines school reading performance in two ways, 
depending on the year under discussion: 

• Through 2013/14 the state measured school performance by aggregating student reading 
scores on the Florida Comprehensive Assessment Test 2.0, as reported previously by 
Folsom et al. (2016). School reading performance is the sum of the percentage of stu- 
dents who scored at or above achievement level 3 (satisfactory) on the reading component 
of the prior year’s FCAT 2.0 and the percentage of students whose scores on the reading 
component of the FCAT 2.0 showed learning gains from the prior year. School reading 
performance through 2013/14 had a theoretical range of 0-200; however, the observed 
range in this study was 54-185. The Florida Department of Education used 2013/14 
school reading performance to identify the 300 lowest performing schools in 2014/15. 

• School reading performance in 2014/15 was equal to the percentage of students who 
achieved a passing score on either the Florida Standards Assessment English language 
arts achievement test or the reading component of the Florida Alternate Assessment. 
School reading performance had a theoretical range of 0-100; however, the observed 
range in this study was 5-96. 

Title I. Title I provides additional resources to schools with economically disadvantaged 
students (Florida Department of Education, n.d.). Schools implementing schoolwide reform 
models provide all students with access to services. In this report these schools are referred 
to as Title I schools. 

Note 

1. When the Florida Department of Education identified the 300 lowest performing schools in 2014/15 on the 
basis of their 2013/14 school reading performance, the 300th school had a school grade of 103. This value, 
which became the cutscore, was shared by 21 schools, and 286 schools had grades below the score, for a 
total of 307 schools with a grade of 103 or lower. Of those 307 schools, 3 closed prior to the 2014/15 school 
year and were excluded from the study. 
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• How did districts and schools incorporate the extra hour of instruction in 2014/15? 
Was there evidence of extra reading instruction? How was the extra time added, 
and how was instruction provided? 

• What were the perceived benefits of and barriers to incorporating the extra hour? 

The data and methodology behind the study are summarized in box 2 and detailed in 
appendix A. 


Box 2. Data and methods 

Data 

The study examined 1,798 elementary schools, 304 of which were identified by the Florida 
Department of Education as the lowest performing schools and required to implement the 
extended school day policy in 2014/15, and 1,494 other elementary schools that were not 
required to implement the extended school day policy in 2014/15. 

The Florida Department of Education published the list of the 300 lowest performing schools 
on its website (2014c). These schools were matched to publicly available School Accountability 
Reports (Florida Department of Education, 2014a), School Grades (Florida Department of Edu- 
cation, 2016b), Membership by School by Grade (Florida Department of Education, 2015), and 
Master School Identification files (Florida Department of Education, 2016c). 

The Florida Department of Education, through the Just Read, Florida! office, developed a 
compliance survey that was distributed electronically and completed by 245 respondents in 
early 2015 (see appendix B for a copy of the survey). In some cases the district completed 
the surveys on behalf of the schools; in other cases the district forwarded the surveys directly 
to schools for completion by the principal or the principal’s designee. The exact proportion of 
responses completed by district personnel is not known. Although 245 surveys (representing 
81 percent of the lowest performing schools) were returned, not every question was answered 
on each survey (see tables C2 and C3 in appendix C for response rates for each question). The 
Florida Department of Education, through the Just Read, Florida! office, also conducted nine 
structured interviews with principals, assistant principals, reading coaches, and other individ- 
uals (that is, a counselor and a curriculum resource teacher) responsible for overseeing imple- 
mentation of the extended school day policy at a random selection of nine lowest performing 
schools (see appendix B for the structured interview protocol). In some cases the interview 
was conducted with only the principal or assistant principal; in other cases the reading coach, 
counselor, or curriculum resource teacher joined the interview. 

Data analysis 

The study team calculated descriptive statistics (means, standard deviations, and cross- 
tabulations) to describe the location, demographic characteristics, and school reading per- 
formance of the lowest performing schools and other elementary schools. Similar descriptive 
statistics were calculated to summarize survey findings on how the lowest performing schools 
implemented the extended school day policy. Tests of statistical significance can be found in 
appendix D. 

In addition, the study team used a qualitative, three-phase approach to code and analyze 
the interview and survey response data. The study team used pattern coding and an itera- 
tive approach to capture constructs and emergent themes in the data that would supplement 
the findings on how the lowest performing schools implemented the policy. The interview and 
survey responses were coded and analyzed within and across districts. 
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What the study found 


This section details the results of the analyses. It describes the location, demographic char- 
acteristics, school reading performance, and implementation of the extended school day 
policy in the lowest performing schools. It also discusses how the findings from the current 
study compare with those of the previous study of the 100 lowest performing schools 
(Folsom et al, 2016). 

The lowest performing elementary schools in reading differed demographically from other 
elementary schools 

This section describes the demographic differences between the lowest performing schools 
and other elementary schools. See tables C2 and C3 in appendix C for descriptive sta- 
tistics for both the aggregated and disaggregated samples. See appendix D for results of 
supplemental tests of statistical significance. 

The lowest performing schools were located in 31 of Florida’s 65 school districts and in 
all five regions of the state (map 1). In each region 15-18 percent of elementary schools 
were identified as lowest performing schools. All regions had schools that were identified 
as lowest performing schools in 2014/15 (9-14 percent of schools in each region) and as 
lowest performing schools in 2014/15 and one prior year (4-7 percent of schools in each 
region). Four of five regions had schools that were identified as lowest performing schools in 
2012/13-2014/15. Across these four regions, 1-2 percent of schools were identified as lowest 
performing schools in 2012/13-2014/15. Only Region 3 (which includes Orlando) did not 
have any schools that were identified as lowest performing schools in 2012/13-2014/15. 

The lowest performing schools were not evenly distributed across regions. Region 1 (the 
northwest region, which includes Tallahassee) had 8 percent of all lowest performing 
schools, whereas region 5 (the southern region, which includes Miami) had 33 percent. 
The differences in the proportion of cohorts between regions were statistically significant. 
These regional differences may be related to the differences in population (and hence in 
number of schools) across regions. For example, Region 1 has 8 percent of other elemen- 
tary schools in Florida, whereas Region 5 has 32 percent of other elementary schools (see 
map 1). 

The lowest performing schools enrolled fewer students than did other schools. Enroll- 
ment averaged 542 students in the lowest performing schools and 660 students in other 
elementary schools (figure 1). The lowest performing schools in 2014/15 averaged 566 stu- 
dents, the lowest performing schools in 2014/15 and one prior year averaged 510 students, 
and the lowest performing schools in 2012/13-2014/15 averaged 474 students. The differ- 
ences in student enrollment were statistically significant across cohorts. Enrollment and 
school reading performance were moderately correlated (r = .22). 

Ninety-nine percent of the 300 lowest performing schools were Title I schools. In con- 
trast, 59 percent of other elementary schools were Title I schools (figure 2). The differences 
in the proportion of Title I schools were statistically significant across cohorts. 

Across the 300 lowest performing schools, on average, 79 percent of students were 
racial/ethnic minority students. In contrast, among other elementary schools, on average, 


In each region 
of Florida 
15-18 percent 
of elementary 
schools were 
identified as lowest 
performing schools 
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Map 1. In 2014/15 the lowest performing elementary schools in reading were located in all five of 
Florida’s regions 



Region 1 

Cohort 

Number 

Share of 
total in 
region 
(percent) 

Share 
of total 
in state 
(percent) 

Other elementary schools 

118 

84 

8 

All lowest performing schools 

23 

16 

8 

Lowest performing schools in 2014/15 

13 

9 

7 

Lowest performing schools in 2014/15 and 
one prior year 

8 

6 

9 

Lowest performing schools in 2012/13- 
2014/15 

2 

1 

9 

Total 

141 

100 

8 


Tampa 


Region 4 

Cohort 

Number 

Share of 
total in 
region 
(percent) 

Share 
of total 
in state 
(percent) 

Other elementary schools 

321 

85 

22 

All lowest performing schools 

58 

15 

19 

Lowest performing schools in 2014/15 

34 

9 

18 

Lowest performing schools in 2014/15 and 
one prior year 

16 

4 

17 

Lowest performing schools in 2012/13- 
2014/15 

8 

2 

36 

Total 

379 

100 

21 


Region 2 

Cohort 

Number 

Share of 
total in 
region 
(percent) 

Share 
of total 
in state 
(percent) 

Other elementary schools 

286 

82 

19 

All lowest performing schools 

62 

18 

20 

Lowest performing schools in 2014/15 

37 

11 

20 

Lowest performing schools in 2014/15 and 
one prior year 

17 

5 

18 

Lowest performing schools in 2012/13- 
2014/15 

8 

2 

36 

Total all schools 

348 

100 

19 



Region 3 

Cohort 

Number 

Share of 
total in 
region 
(percent) 

Share 
of total 
in state 
(percent) 

Other elementary schools 

298 

83 

20 

All lowest performing schools 

61 

17 

20 

Lowest performing schools in 2014/15 

51 

14 

27 

Lowest performing schools in 2014/15 and 
one prior year 

10 

3 

11 

Lowest performing schools in 2012/13- 
| 2014/15 

0 

0 

0 




Si- 


Region 5 

| Cohort 

Number 

Share of 
total in 
region 
(percent) 

Share 
of total 
in state 
(percent) 

Other elementary schools 

471 

83 

32 

All lowest performing schools 

100 

18 

33 

Lowest performing schools in 2014/15 

54 

10 

29 

Lowest performing schools in 2014/15 and 
one prior year 

42 

7 

45 

Lowest performing schools in 2012/13- 
2014/15 

4 

1 

18 

Total 

571 

100 

32 


Source: Authors' analysis of data from Florida Department of Education (2014c, 2016c). 


56 percent of students were a racial/ethnic minority (see figure 2). The average percent- 
age of racial/ethnic minority students was 77 percent in the lowest performing schools in 
2014/15 and 89 percent in both the lowest performing schools in 2014/15 and one prior 
year and the lowest performing schools in 2012/13-2014/15. The differences in the per- 
centage of racial/ethnic minority students were statistically significant across cohorts. 

Across the 300 lowest performing schools, on average, 76 percent of students were eli- 
gible for the federal school lunch program. In contrast, the percentage was 59 percent in 
other elementary schools (see figure 2). The average percentage of students eligible for the 
federal student lunch program was 73 percent in the lowest performing schools in 2014/15, 
80 percent in the lowest performing schools in 2014/15 and one prior year, and 79 percent 
in the lowest performing schools in 2012/13-2014/15. The differences in the percentage of 
students eligible for the federal school lunch program were statistically significant across 
cohorts. 

The lowest performing elementary schools had significantly lower school reading performance, on 
average, than did other elementary schools 

As would be expected, the average school reading performance of the lowest performing 
schools was significantly lower than that of the state’s other elementary schools (figure 3). 


The average 
percentage of 
students eligible 
for the federal 
student lunch 
program was 
73 percent 
in the lowest 
performing schools 
in 2014/15, 

80 percent in the 
lowest performing 
schools in 
2014/15 and one 
prior year, and 
79 percent in the 
lowest performing 
schools in 
2012/13-2014/15 
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Figure 1. On average, enrollment was lower in Florida’s lowest performing 
elementary schools in 2014/15 than in other elementary schools 


Number of students 
750 


500 - 



250 - 


Other All lowest Lowest performing Lowest performing Lowest performing 

elementary performing schools in schools in schools in 

schools schools 2014/15 2014/15 and 2012/13- 

(n = 1,494) (n = 304) (n = 189) one prior year 2014/15 

( n = 93) (n = 22) 


Source: Authors’ analysis of data from Florida Department of Education (2015). 


Figure 2. Florida’s lowest performing elementary schools in 2014/15, on average, 
enrolled higher proportions of racial/ethnic minority students and students eligible 
for the federal school lunch program than did other elementary schools across the 
state 


■ Share of schools that are Title I schools 

■ Share of students who are a racial/ ethnic minority 

■ Share of students eligible for the federal school lunch program 



Other 

elementary 
schools 
(n = 1,494) 


All lowest 
performing 
schools 
(n = 304) 


Lowest performing Lowest performing 
schools in schools in 

2014/15 2014/15 and 

(n = 189) one prior year 

(n = 93) 


Lowest performing 
schools in 
2012/13- 
2014/15 
(n = 22) 


Source: Authors' analysis of data from Florida Department of Education (2016b). 
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Figure 3. On average, school reading performance in 2013/14 and 2014/15 was 
lower in Florida’s lowest performing schools than in other elementary schools 


■ Other elementary schools (n = 1,494) 

■ All lowest performing schools (n = 304) 

■ Lowest performing schools in 2014/15 (n = 189) 

_ , , , ■ Lowest performing schools in 2014/15 and one prior year (n = 93) 

School reading performance (z-score) Lowest performing schools in 2012/13-2014/15 (n = 22) 

0.5 -| 



a. 2013/14 school reading performance is based on reading scores on the Florida Comprehensive Assess- 
ment Test (FCAT) 2.0, and 2014/15 school reading performance is based on Florida Standards Assessment 
English language arts achievement scores. Because the two tests uses different scales, scores were trans- 
formed into z-scores with a mean of 0 and standard deviation of 1. 

Source: Authors' analysis of data from Florida Department of Education (2014a, 2016b). 


Average 2013/14 school reading performance for all lowest performing schools was 92.4, 
which was 1.8 standard deviations below the average for other elementary schools (133.9). 
The lowest performing schools in 2014/15 had the highest average school reading perfor- 
mance (94-6), followed by the lowest performing schools in 2014/15 and one prior year 
(90.7) and the lowest performing schools in 2012/13-2014/15 (81.2). Overall, the differ- 
ences in school reading performance among the four cohorts (lowest performing schools 
in 2014/15, lowest performing schools in 2014/15 and one prior year, lowest performing 
schools in 2012/13-2014/15, and other elementary schools) was statistically significant. The 
school reading performance of all cohorts of lowest performing schools was statistically 
significantly lower than that of other elementary schools. The school reading performance 
of the lowest performing schools in 2014/15 was higher than that of the lowest performing 
schools in 2012/13-2014/15. 


Average 2013/14 
school reading 
performance 
for all lowest 
performing schools 
was 92.4, which 
was 1.8 standard 
deviations below 
the average for 
other elementary 
schools (133.9) 


Average 2014/15 school reading performance for all lowest performing schools was 32.1, 
which was 1.5 standard deviations below that of other elementary schools (mean of 56.5), 
after implementing the extended school day policy in 2014/15 or, in some cases, for mul- 
tiple years between 2012/13 and 2014/15. The lowest performing schools in 2014/15 had 
the highest school reading performance (mean of 35.1), followed by the lowest performing 
schools in 2014/15 and one prior year (mean of 28.5) and the lowest performing schools in 
2012/13-2014/15 (mean of 21.2). Overall, the differences in school reading performance 
among the four cohorts (lowest performing schools in 2014/15, lowest performing schools 
in 2014/15 and one prior year, lowest performing schools in 2012/13-2014/15, and other 
elementary schools) was statistically significant. The school reading performance of all 
cohorts of lowest performing schools was statistically significantly lower than that of other 
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elementary schools. The school reading performance of the lowest performing schools in 
2014/15 was higher than that of the lowest performing schools in 2014/15 and one prior 
year and that of the lowest performing schools in 2012/13-2014/15. 

While a direct comparison of 2013/14 school reading performance and 2014/15 school 
reading performance is not possible because different assessments were used in each year, 
the standardized mean difference between the lowest performing schools and other ele- 
mentary schools was smaller in 2014/15 than in 2013/14. This suggests that the gap between 
the lowest performing schools and other elementary schools may be closing. 

The lowest performing schools reported using a variety of strategies to comply with the extended 
school day policy 

In general, the survey responses of the lowest performing schools in 2014/15, the lowest 
performing schools in 2014/15 and one prior year, and the lowest performing schools in 
2012/13-2014/15 were not significantly different. Unless otherwise noted, this section 
reports on survey responses of all lowest performing schools. See tables C2 and C3 in 
appendix C for descriptive statistics for both the aggregated and disaggregated samples. 
See appendix D for results of supplemental tests of statistical significance. 

Schools reported increasing reading instruction time each day. The average amount of 
reading instruction each day that all lowest performing schools reported providing was 144 
minutes in 2013/14 and 181 minutes in 2014/15. The increase was statistically significant. 
The differences in the average amount of instruction and in the average increase in the 
amount of instruction were statistically significant across cohorts (figure 4). 

The lowest performing schools in 2014/15 reported providing an average of 128 minutes of 
reading instruction a day in 2013/14 and 180 minutes in 2014/15. The lowest performing 
schools in 2014/15 and one prior year reported providing an average of 161 minutes of 
reading instruction a day in 2013/14 and 182 minutes in 2014/15, a statistically significant 
increase of 22 minutes. Because these schools had previously been required to extend the 
school day in 2012/13 or 2013/14, the overall mean increase was less than a full hour. Six- 
ty-seven schools were already implementing the extended school day policy in 2013/14, 
and some of the 72 that were required to implement it in 2012/13 may have continued 
to incorporate the extra hour in 2013/14 even if they were no longer required to do so. 
However, others may have discontinued the extra hour in 2013/14. For the lowest perform- 
ing schools in 2014/15 and one prior year that were also identified as lowest performing 
schools in 2012/13 but not 2013/14, the average reported increase was 34 minutes (from 153 
minutes to 186; the difference does not match the total because of rounding). The lowest 
performing schools in 2014/15 and one prior year that were also identified as lowest per- 
forming schools in 2013/14 but not in 2012/13 (meaning that they were identified as lowest 
performing schools in consecutive years), reported an average increase of 7 minutes (from 
170 minutes to 178; the difference does not match the total because of rounding). 

The lowest performing schools in 2012/13-2014/15 reported that schools provided an 
average of 183 minutes of reading instruction a day in 2013/14 and 184 minutes in 2014/15. 
This minimal increase is expected, as these schools had been required to implement the 
extra hour in 2012/13, 2013/14, and 2014/15. 
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Figure 4. Florida’s lowest performing schools in 2014/15 reported the largest 
increase in the amount of reading instruction provided in 2014/15 


— Lowest performing schools in 2014/15 
(2013/14: n = 125; 2014/15: n = 135) 

— Lowest performing schools in 2014/15 and one prior year 
(2013/14: n = 81; 2014/15: n = 81) 

Minutes of reading instruction Lowest performing schools in 2012/13-2014/15 

2Qo (2013/14: n = 20; 2014/15: n =22) 


180 - 


160 - 


140 - 


120 



2013/14 


2014/15 


Note: Information about the amount of instruction provided in 2013/14 was retroactively reported in the 
2014/15 survey. Information about the amount of instruction provided in 2013/14 was not reported on every 
survey. 

Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day compliance survey (see appendix B). 
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Schools reported using a variety of strategies to incorporate the extra hour of instruc - 
tion. Survey respondents representing 244 schools elaborated on how the extra hour was 
incorporated into the instructional day. The differences in the percentage of schools that 
used each strategy to incorporate the extra hour were not statistically significant across 
cohorts (figure 5). The most common strategy was to end the school day later (34 percent), 
followed by ending the school day later and rearranging the instructional day (25 percent). 
For example, one survey respondent reported that the school added 15 minutes to the 
beginning of the day, added 15 minutes to the end of the day, shortened enrichment time, 
and eliminated lab time to accommodate the extra hour of instruction. Another reported 
that the entire district extended the school day to incorporate a districtwide, one-hour 
intervention block. One survey respondent described a midyear change for incorporating 
the hour: “After one semester of scheduling the additional hour at the end of the day, the 
leadership team . . . readjusted the schedule to incorporate the additional 60 minutes into 
the regular day.” 

Schools reported using a combination of students’ regular classroom teachers and other 
staff to provide the extra hour of instruction. The differences in the percentage of schools 
that used each type of staff to provide the extra hour of instruction were not statistically 
significant across cohorts. That is, all cohorts of lowest performing schools had similar 
proportions of regular teachers, other staff, and combinations of regLilar teachers and other 
staff providing the extra hour of instruction. Eighty-three percent of schools reported using 
a combination of regular classroom teachers and other staff to provide the extra hour of 
instruction, 16 percent reported using only the students’ regular teachers, and 1 percent 
reported using only other staff (figure 6). 
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Figure 5. The most common strategy that Florida’s lowest performing schools 
that implemented the extended school day policy in 2014/15 reported using to 
incorporate the extra hour of instruction was to end the school day later 


Percent of schools 
40 -I 



Start school End school Other Rearrange Start school Start school End school Start school 
earlier later instructional earlier and earlier and later and earlier, end 

day end school rearrange rearrange school later, 
later instructional instructional and rearrange 
day day instructional 

day 


Note: Data are for 244 schools. 

Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day compliance survey (see appendix B). 


Figure 6. Florida’s lowest performing schools that implemented the extended 
school day policy in 2014/15 reported using a combination of students’ regular 
classroom teachers and other staff to provide the extra hour of reading instruction 


Other staff 1% 



Note: Data are for 221 schools. 

Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day compliance survey (see appendix B). 
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The interview and survey responses revealed that other staff were used mainly to provide 
instructional support for teachers during the extra hour. Other staff included administra- 
tors, resource staff, media specialists, guidance counselors, academic support staff, English 
learner teachers, special education teachers, volunteers, enrichment teachers, reading 
resource teachers, tutors, teaching assistants, and special area teachers. One respondent 
indicated that teachers from other schools were used to provide instruction during the 
additional hour. 

Support from other staff also included pushing in or pulling out small groups and tiered 
instruction, substituting for teachers when needed, and providing intensive reading 
instruction as required by the policy. For example, a respondent explained, 

Every seventh day of school, classroom teachers have a scheduled planning period 
during the extra hour. This time is intended especially for collaboration on best 
instructional practices for reading. On those days, the reading extra hour is taught 
by special area teachers. 

Other staff substitutions involved teachers who did not meet the school’s standard for 
reading instructor quality and teachers who elected to have someone else provide the 
instruction. 

Schools reported adding staff to meet the increased instructional needs. Survey respons- 
es showed that a larger percentage of schools increased rather than reduced the number of 
reading coaches, teachers, paraprofessionals, volunteers, or other staff (counselors, media 
specialists, interventionists, and teacher-leaders; figure 7). 

Differences in staffing changes that involved coaches, teachers, paraprofessionals, and 
other staff were statistically significant across cohorts. For example, the percentage of 
schools that reported adding reading coaches was higher among the lowest performing 
schools in 2014/15 (64 percent) than among the lowest performing schools in 2014/15 and 
one prior year (32 percent) and among the lowest performing schools in 2012/13-2014/15 
(11 percent). The percentage of schools that reported reducing the number of teachers 
was higher among the lowest performing schools in 2012/13-2014/15 (29 percent) than 
among the lowest performing schools in 2014/15 (3 percent) and among the lowest per- 
forming schools in 2014/15 and one prior year (16 percent). The percentage of schools that 
reported adding paraprofessional staff was higher among the lowest performing schools 
in 2014/15 (64 percent) than among the lowest performing schools in 2014/15 and one 
prior year (38 percent) and the lowest performing schools in 2012/13-2014/15 (47 percent). 
The proportion of schools that reported reducing the number of other staff was 0 percent 
among the lowest performing schools in 2014/15, 14 percent among the lowest performing 
schools in 2014/15 and one prior year, and 6 percent among the lowest performing schools 
in 2012/13-2014/15. 

Interview participants highlighted changes in leadership when asked about staffing 
changes. Five of the nine interviewees reported having the same principal for fewer than 
two years between 2012/13 and 2014/15. Four of the nine schools were reportedly under 
new leadership during the implementation period. One interviewee whose school had 
had seven principals in 18 years cited a need for more consistency in leadership. Another 
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Figure 7. The percentage of schools that reported adding staff in 2014/15 
was higher among Florida’s lowest performing schools in 2014/15 than among 
the lowest performing schools in 2014/15 and one prior year and the lowest 
performing schools in 2012/13-2014/15 
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Note: The n values refer to the number of schools that responded to the survey question on changes in each 
type of staffing. For example, the first bar indicates that 178 of all lowest performing schools provided informa- 
tion on changes in reading coach staffing. 

Source: Authors’ analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day compliance survey (see appendix B). 


school had reportedly had a complete changeover in leadership — the principal, assistant 
principal, dean, guidance counselor, and special education staff specialist were all new. 

Schools reported providing professional development or training to support staff involved 
in reading instruction. In the analysis of the open responses to the surveys, 66 schools 
provided information related to the professional development and training teachers 
received. The types and combinations of professional development varied. Examples cited 
by survey respondents included professional development administered through bi-weekly 
professional learning communities and professional development opportunities designed to 
develop highly effective reading teachers. 
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Interviewees from eight of the nine schools indicated that schools offered one or more 
types of professional development to support the teachers who were providing the extra 
hour of instruction. Six of these schools offered one-time training on a specific curriculum 
or software. Interviewees also reported that schools offered ongoing professional develop- 
ment either on the specific curriculum (four schools) or broadly through coaching (five 
schools), cooperative learning (one school), or professional learning communities (two 
schools). Interviewees gave specific examples, such as having “professional development 
Mondays” for team planning and using a gradual-release model of professional develop- 
ment. One interviewee reported that ongoing professional development included training 
on aligning activities with standards, common planning facilitated by the literacy coach, 
and training in a reading intervention. Another school reportedly surveyed teachers to 
identify their professional development needs, which district administrators then addressed 
by facilitating access to appropriate training. 
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Schools reported using a variety of grouping strategies to incorporate the extra hour of 
reading instruction. While it was not possible for the study team to determine whether 
schools provided differentiated instruction as required by the extended school day policy, 
the fact that schools reported using small groups of students sorted by ability level for the 
extra hour of instruction suggests that schools may have attempted to provide differenti- 
ated instruction. There were no statistically significant differences in grouping of students 
for instruction across cohorts. 


Nearly all schools reported using small-group instruction either exclusively or in combina- 
tion with large-group instruction (figure 8). Two percent of schools reported using large- 
group instruction exclusively. Nearly all schools reported grouping students for instruction 
by ability either exclusively or in combination with grouping students with mixed abilities 
(figure 9). Seven percent of schools reported only grouping students with mixed abilities. 

Of the 131 survey respondents who provided information on whether grouping dif- 
fered between the regular reading block and the extra hour of instruction, 106 reported 


Figure 8. Nearly all of Florida’s lowest performing schools reported using small- 
group reading instruction either exclusively or in combination with large-group 
instruction in 2014/15 


Large group 2% 



Note: Data are for 231 schools. 

Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day compliance survey (see appendix B). 
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Figure 9. Nearly all of Florida’s lowest performing schools schools reported 
grouping students for instruction by ability either exclusively or in combination with 
grouping students with mixed abilities in 2014/15 


Grouping students 
with mixed abilities 
7% 



Note: Data are for 231 schools. 

Source: Authors’ analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day compliance survey (see appendix B). 


different grouping. While respondents’ descriptions of groupings were largely consistent 
with the survey’s overall findings (for example, whole group, small groups, grouping based 
on ability), it was not possible to ascertain specific grouping differences between regular 
reading instruction and the extra hour. That is, respondents affirmed that grouping difi 
fered, but when describing the differences, they did not indicate specifically which group- 
ing was used in the extra hour. 

The interview responses were also consistent with the survey findings. Only two of the 
nine interviewees reported that they did not group students differently for regular instruc- 
tion and the extra hour. Interviews revealed that the other seven schools generally used 
data to group students according to their needs. The criterion used most often to group 
students was progress-monitoring data (five of seven schools), followed by teacher recom- 
mendations (three of seven schools). Other criteria included various individual assessments 
(three of seven schools reported using at least one type of assessment) and random assign- 
ment (one of seven schools). 


Schools reported complying with the extended school day policy. Compliance with the 
policy was based on eight criteria. A majority of survey respondents (83-99 percent across 
criteria) reported meeting or exceeding each criterion for compliance (figure 10). The 
weakest reported area of compliance was using teachers with evidence of prior success 
in teaching reading. The strongest reported areas of compliance were providing research- 
based instruction and using assessment and progress monitoring to determine instructional 
priorities. Except in the areas of parent notification and research-based curriculum, there 
were no statistically significant differences in compliance across cohorts. The lowest per- 
forming schools in 2012/13-2014/15 were less likely than the lowest performing schools in 
2014/15 and the lowest performing schools in 2014/15 and one prior year to report exceed- 
ing the criterion for parent notification or for using a research-based curriculum. 
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Figure 10. A majority of schools reported meeting or exceeding each of the eight 
criteria for compliance with Florida’s extended school day policy in 2014/15 


■ Does not meet criterion ■ Partially meets criterion Meets criterion ■ Exceeds criterion 


Notify parents (n = 245) 
FCAT level 5 flexibility (n = 238) a 
Teacher evidence of prior success ( n = 242) 
Research-based instruction (n = 244) 
Research-based curricula (n = 242) 
Assessment and progress monitoring (n = 235) 
Regular monitoring (n = 235) 
Differentiated instruction (n = 241) 



Percent 


FCAT = Florida Comprehensive Assessment Test 2.0. 

Note: The n values refer to the number of schools that responded to the survey question on each criterion. For 
example, the first bar indicates that 245 schools provided information on whether they notified parents that 
their student had been enrolled in an extra hour of reading instruction each day. 

a. Schools were given the flexibility to allow students scoring at Level 5 to opt out of the extended reading 
instruction. Level 5 is the highest achievement level and indicates that a “student has success with the most 
challenging content of the Sunshine State Standards. A student scoring in Level 5 answers most of the test 
questions correctly, including the most challenging questions” (Florida Department of Education, 2008). 

Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day policy compliance survey (see appendix B). 


Interviewees also described a variety of instructional approaches that were part of com- 
pliance criteria. For example, interviewees from seven schools mentioned differentiated 
instruction, and three mentioned research-based instruction. One interviewee discussed 
integrating phonemic awareness, phonics, fluency, vocabulary, and comprehension instruc- 
tion. Two interviewees described integrating other subjects with reading instruction. 

Participants viewed the extended school day policy as beneficial and easier to implement over time 
and noted increased professional development and curricular and pedagogic changes as indirect 
benefits of implementation 

Six of seven interviewees who reported on overall perceptions had a positive attitude 
toward the extra-hour requirement. However, five of the six interviewees who reported on 
staff enthusiasm said that staff were not enthusiastic about implementing the additional 
hour of instruction. Two interviewees reported that it was not until after the first year of 
implementation that the staff of their schools saw the benefits of the policy. These two 
interviewees also highlighted that incorporating the extra hour had become easier over 
time. One of these interviewees who had previously incorporated the extra hour discussed 
that the school was more prepared and “more organized than [the] year before.” Interview- 
ees from two of the six lowest performing schools in 2014/15 indicated that they “got better 
[at implementing the policy] over the course of the year”; an interviewee from one of those 
schools reported not initially having the necessary instructional materials because of the 
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late notice. Interviewees from two of six schools mentioned that they felt “more prepared 
for 2015/16,” possibly because their interviews were conducted after schools were notified 
that they would continue implementing the policy in 2015/16. 

Overall, the interviewees found the extra hour beneficial because it allowed more time for 
instruction (five of the nine schools) and increased opportunities to meet students’ needs 
(five of the nine schools). 

Professional development emerged as the main schoohlevel support for implementing the 
extra hour of instruction. Interviewees from six of the nine schools reported that their 
school provided professional development or planning time for teachers; interviewees from 
five of the nine schools saw this as a benefit of the extended school day policy. But inter- 
viewees from two of the nine schools cited lack of professional development and loss of 
planning time as barriers to implementing the extra hour of instruction. One of those 
interviewees stated that “special area teachers (for example, physical education, music) did 
not feel as prepared for this instruction.” This may not have been a challenge for schools 
that used only certified teachers to implement the extra hour. 
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Interviewees at eight schools responded to a question about district support for implemen- 
tation. Two interviewees reported district support from the beginning, and one of them 
noted that the district “provided support while allowing principal autonomy for school 
implementation.” However, the majority of interviewees (five of the eight that reported on 
perceived district support) had negative perceptions of district support. One interviewee 
described feeling “very negative about resources available to this school and the lack of 
support for implementation. [We] also felt the district was not listening to educators and 
school-based staff in making decisions.” 


Interviewees had a positive attitude toward the extra hour requirement. All interviewees 
expressed observing individual student gains (despite lack of empirical evidence to support 
these claims) over the year in which the extra hour was incorporated. But interviewees 
from only one school specifically attributed the perceived gains to the extra hour. Other 
interviewees attributed the perceived gains to curricular or pedagogical changes (six of 
nine schools), common or collaborative planning time (two of nine schools), extra time 
on task for students (one of nine schools), and afterschool tutoring (one of nine schools). 
Examples of curricular or pedagogical changes included using data to inform instruction 
(three of six schools), differentiating instruction (five of six schools), using direct instruc- 
tion (one of six schools), using instructional coaching (three of six schools), enhancing 
core instruction (one of six schools), and changing how students were grouped on the 
basis of data (six of six schools). Other factors that interviewees believed helped students 
improve were surveying and observing staff to ensure that teachers’ needs were being met, 
including their needs for materials (three of nine schools); changing the culture and rela- 
tionships within the classroom (one of nine schools); and providing stipends to teachers 
(one of nine schools). Two interviewees also indicated that professional development and 
coaching — for example, leadership training; coaches and administrators modeling, observ- 
ing, and providing feedback; professional learning communities; and reflecting on chal- 
lenges in the classroom — contributed to student achievement. Even though interviewees 
attributed their perceived rise in student test scores to pedagogy and professional develop- 
ment, it is not possible to rigorously distinguish the effects of professional development and 
pedagogy from the effects of the extended school day. 
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Despite the noted benefits of implementation, all interviewees identified barriers to imple- 
mentation, including a lack of resources, funding, or technology (two of the nine schools); 
difficulty acquiring support staff and personnel (two of the nine schools); lack of staff buy-in 
(one of the nine schools); and taking time away from other subject areas (one of the nine 
schools). Interviewees from three of the nine schools identified the short notice, schedule 
changes, and lack of district guidance as barriers. One of the interviewees also noted that 
the state was implementing new learning standards in 2014/15 that entailed additional chal- 
lenges, but this interviewee did not specify what those challenges were. Interviewees were 
not asked, nor did they provide clarification on, why they perceived these issues as barriers. 

The findings mirror those in previous research on Florida’s 100 lowest performing schools 

This study follows up on a previous study of the extended school day policy in Florida’s 
100 lowest performing schools that was in effect from 2012/13 through 2013/14 (Folsom 
et al., 2016). Generally, the findings were consistent with the previous study. This section 
highlights notable differences between the findings of the previous study and those of 
the current study. Both studies found that a combination of small-group and large-group 
instruction was used to provide the extra instruction. However, the percentage of survey 
respondents that reported using small-group instruction exclusively was greater in the 
current study (38 percent) than in the previous study (28 percent). Both studies found 
that the majority of schools used a combination of students’ regular teachers and other 
staff. But the current study found that 16 percent of survey respondents reported using stu- 
dents’ regular teachers exclusively, compared with 29 percent of respondents to the survey 
in the previous study. While both studies found that schools hired additional staff, fewer 
respondents reported hiring additional staff in the survey in the current study than in the 
survey in the previous study. This discrepancy may reflect the changes in the language of 
the law that provided more flexibility in who was qualified to provide the extra instruction, 
although that speculation cannot be verified. 

Implications of the study findings 


All school staff reported positive outcomes after implementing the policy. Staff observed 
individual student gains during the year the extended school day policy was implemented, 
although this could not be documented in the empirical analyses. Interviewees attribut- 
ed these perceived individual student gains to curricular or pedagogical changes that 
accompanied implementation of the policy. Across the survey and interviews, two themes 
emerged that have implications for future implementation of the extended school day 
policy: 

• Districts and schools may benefit from working together to implement the policy 
while providing schools a degree of autonomy in the process. For example, districts 
could survey teachers and administrators to determine what professional devel- 
opment, materials, technology, and other support they need to provide the extra 
instruction. Support and communication about why the requirement is in place 
may boost staff morale and buy-in. 

• Because respondents reported using other staff to provide instruction, schools 
appear to be relying on teacher supervision of those other staff. Because no offi- 
cial guidance exists on what makes a teacher “effective in teaching reading,” the 
Florida Department of Education may want to consider providing such guidelines, 
particularly to those who are choosing supervisors of other staff. 
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Limitations of the study 


This study was designed to describe how schools implemented the extended school day 
policy in Florida’s 300 lowest performing elementary schools in 2014/15. This study was 
not designed to determine whether the policy was effective. Similarly, this study was not 
designed to identify the best way to incorporate the extra hour of instruction or which 
aspects of policy implementation were effective. Such objectives could be accomplished 
only through an appropriate study design that allows for the study of causal relations, such 
as a randomized controlled trial or a regression discontinuity design. This may be a conskh 
eration for future research. 


The survey data may suffer from inaccuracies. The Florida Department of Education 
requested the district surveys as a compliance check, and thus respondents may have felt a 
high social desirability to provide the “right” answers that would make their schools appear 
compliant with the policy. 

The surveys were not formally validated, and the survey tool may be more aptly referred to 
as a questionnaire. Incorporating validated items on specific aspects of the extended school 
day policy or on context for implementation could provide more accurate data. Validation 
would also enable the Florida Department of Education to assess whether the survey items 
for each construct validly measure what they are being used to measure and to what extent 
they are internally consistent. 

Although the survey and interview protocol (see appendix B) used terminology that should 
be familiar to Florida educators (for example, “comprehensive intervention program” in 
question 5), some questions may have been confusing. Each compliance survey item includ- 
ed an expectation statement from the Florida Department of Education along with the 
relevant statutory requirement as written in House Bill 5101, and these prompts were not 
always clearly aligned. In questions 1-8 it may have been unclear to respondents whether 
they were responding to the statement as written by the Florida Department of Education 
or to the alignment of statutory requirements. For example, question 8 gives the state- 
ment from the Florida Department of Education, “Instruction must be differentiated and 
include: small group instruction based on student assessment data to meet students’ spe- 
cific reading needs and include formative assessment with writing, and extensive reading 
from a wide variety of texts, all verified with data logs,” while the statutory requirement 
says, “The intensive reading instruction delivered in this extra hour and for other students 
shall include: differentiated instruction based on student assessment data to meet students’ 
specific reading needs.” Some questions may also seem contradictory. For example, ques- 
tion 1 asks about notification to all parents that all students will have an extended school 
day, but question 2 asks about providing an exemption for students who are performing 
above grade level. 
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The Just Read, Florida! office of the Florida Department of Education anecdotally informed 
the study team that generally staff at districts, not schools, completed the surveys. Thus 
the respondents may not have known the nuances of how the extended school day policy 
was implemented at each school. Without triangulating the survey data with direct obser- 
vations, the survey results on implementation are limited. 


18 


Interviews were used to gain additional insight into how schools implemented the extra 
hour of instruction, but the interview sampling and methods present limitations. Inter- 
views were conducted in only 3 percent of the lowest performing schools: of the nine inter- 
views, six were with staff from the lowest performing schools in 2014/15 and three were 
with staff from the lowest performing schools in 2014/15 and one prior year; no interviews 
were conducted with staff from the lowest performing schools in 2012/13-2014/15. A larger 
sample that included interviews with staff from the lowest performing schools in 2012/13- 
2014/15 would have provided a more complete picture of all the lowest performing schools. 
Future research may consider using stratified random sampling to ensure a more represen- 
tative sample of districts in which to conduct interviews. While interviews provide more 
detail and deeper insights into participants’ experiences — and into how they make sense 
of those experiences (McMillan & Schumacher, 2010) — the data are self-reported and rely 
on accurate and honest reflections or accounts from the interviewees. The quality of the 
information obtained during an interview also depends on the interviewer. For this study, 
only the interviewers’ written notes on the structured interview protocol were available 
for analysis. In future studies, audio recordings of the interviews, with full transcriptions, 
could be considered. Lastly, as with the surveys, the interviews were conducted by the 
Florida Department of Education, which could have produced a high social desirability 
bias to provide the “right” answers. 

It is not possible to determine whether the lowest performing schools are doing anything 
different from other schools. Data are unavailable for other schools that have, for example, 
implemented similar extended school day policies or found alternative ways to increase 
reading instruction. This is particularly possible because under Florida House Bill 5101 
reading coaches could be added to the staff only if all students were provided an extra hour 
of reading instruction. Moreover, at least one respondent stated that all schools in the 
district were extending the school day, not just the lowest performing schools. 

The study did not measure quality of instruction. Surveys provide limited information on 
day-to-day operations at a school and limited insight into actual instructional practices. 
The surveys captured information about the extended school day policy only. The lowest 
performing schools could have substantially changed their entire reading instruction 
program beyond simply adding an extra hour of instruction. Future research could conduct 
in-depth classroom observations during both the regular school day and the extra hour. 
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Appendix A. Data and methodology 


This appendix describes the data and methodology used in the study. 

Data 

Included schools. The study sample includes only schools eligible to be designated a lowest 
performing school in all three years of the study (that is, they were open, general education 
elementary schools receiving school grades from the Florida Department of Education in 
2012/13-2014/15). Schools with a special school function or setting (such as virtual, hospi- 
tal/homebound, and Department of Juvenile Justice schools) were excluded. School grades 
are assigned by the Florida Department of Education and communicate how well a school 
is performing relative to state standards. In 2013/14, school grades were assigned primar- 
ily on the basis of aggregated student achievement data from the Florida Comprehensive 
Assessment Test (FCAT) 2.0 (Florida Department of Education, 2014b). In 2014/15, school 
grades were assigned primarily on the basis of aggregated student achievement data from 
the Florida Standards Assessment (Florida Department of Education, 2016a). The first year 
a school is open, it is exempt from grading; thus all schools in their first year of operation 
were excluded from the study. Alternative schools and special education center schools, 
which receive school improvement ratings rather than school grades, were also excluded. 
More information about Florida’s school grades can be found at http://schoolgrades.fldoe.org. 

Lowest performing schools. In 2014/15 the Florida Department of Education (2014c) pub- 
lished the list of the lowest performing schools along with an indication of whether they 
had been designated as a lowest performing school in a previous year. The Florida Depart- 
ment of Education identified the lowest performing schools on the basis of school reading 
performance. School reading performance is the sum of the percentage of students at each 
school scoring at or above achievement level 3 (satisfactory) on the reading component of 
the prior year’s FCAT 2.0 and the percentage of students making learning gains from the 
prior year on the reading component of the FCAT 2.0. 

While the goal of the law is to identify the 300 lowest performing schools, more than 
300 schools can be identified if multiple schools are at the cutscore for the 300th school. 
When the Florida Department of Education identified the 300 lowest performing schools 
in 2014/15 on the basis of their 2013/14 school reading performance, the 300th school had 
a school grade of 103. This value, which became the cutscore, was shared by 21 schools, 
and 286 schools had grades below the cutscore, for a total of 307 schools with a grade of 
103 or lower. Of those 307 schools, 3 closed prior to the 2014/15 school year and were 
excluded from the study. Thus, 304 schools were included in the study and are collectively 
referred to as the 300 lowest performing schools. 

Publicly available data. The list of the 300 lowest performing schools was matched with 
publicly available data files: School Accountability Reports (Florida Department of Edu- 
cation, 2014a), School Grades (Florida Department of Education, 2016b), Membership by 
School by Grade (Florida Department of Education, 2015), and Master School Identifica- 
tion (Florida Department of Education, 2016c). 

The School Accountability Reports (Florida Department of Education, 2014a) and School 
Grades (Florida Department of Education, 2016b) files provide detailed information about 
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student achievement and other components of school grades. These files supplied the fol- 
lowing information pertinent to this study: 

• District and school identifiers, used to link files. 

• School type, used to select schools for analysis. 

• Percentage of students eligible for the federal school lunch program, used to 
describe school composition. 

• Percentage of racial/ethnic minority students, used to describe school composition. 

• School grades, used to identify schools eligible for analysis. 

• Region, used to describe the distribution of the lowest performing schools. 

• Title I status, used to describe the schools. 

• Percentage of students reading at a satisfactory level or higher, used to compute 
school reading performance (2013/14 only). 

• Percentage of students making learning gains in reading, used to compute school 
reading performance (2013/14 only). 3 

• English language arts achievement test scores (2014/15 only). 

The Membership by School by Grade file (Florida Department of Education, 2015) is 
updated yearly by the Florida Department of Education to provide total student enrollment 
by grade and across grades for all Florida schools. This file was matched with the School 
Accountability Reports (Florida Department of Education, 2014a) and School Grades 
(Florida Department of Education, 2016b) files. The file supplied the following information 
pertinent to this study: 

• District and school identifiers, used to link files. 

• Total membership (number of students enrolled in the school). 

The Master School ID file (Florida Department of Education, 2016c) contains up-to-date 
identification and directory information on all Florida public schools. This file was matched 
with the School Accountability Reports (Florida Department of Education, 2014a), School 
Grades (Florida Department of Education, 2016b), and Membership by School by Grade 
(Florida Department of Education, 2015) files to verify schools’ type, region, and Title I 
status from the School Accountability Reports. The files supplied the following informa- 
tion pertinent to this study: 

• District and school identifiers, used to link files. 

• School type, accountability type, and grade configuration variables, used to verify 
that only elementary schools were included in the analysis. 

• Primary service type and school function and setting variables, used to verify that 
only K-12 general education schools were included in the analysis. 

• Activity code, used to determine whether a school was active (open) or closed. 

Extended school day compliance survey. The Just Read, Florida! office of the Florida Depart- 
ment of Education, which oversees implementation of the extended school day policy, dis- 
tributed a compliance survey to district personnel in 2014/15 to gather information on how 
the lowest performing schools were implementing the extra hour. In some cases the district 
personnel completed the surveys on behalf of the schools; in other cases the surveys were 
forwarded to school administrators. It is unknown which personnel completed which surveys. 

Neither the study team nor the state piloted or validated the survey — that is, neither group 
verified or otherwise triangulated the accuracy of the information provided in the surveys. 
It is unknown whether school personnel had input or otherwise verified the responses 
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when district personnel completed the surveys or whether district personnel had input or 
otherwise verified the responses when school personnel completed the surveys. 

The Florida Department of Education provided the original surveys as electronic spread- 
sheets for each school. The study team double-blind entered the information from the 
surveys into a database for analysis. 

Structured interview protocol. At the request of the Regional Educational Laboratory (REL) 
Southeast Improving Literacy Alliance and Improving Low-Performing Schools Alliance, 
the study team provided technical analytic support to develop a scripted structured interview 
protocol, along with a training workshop, for Just Read, Llorida! staff to interview key person- 
nel at schools implementing the extended school day policy (see appendix B). The interview 
protocol was used to guide the interview, take notes, and code interview responses. Regional 
Educational Laboratory Southeast led a workshop to train the Just Read, Llorida! staff on 
collecting qualitative data for research purposes and conducting interviews. The workshop 
also included specific instructions for using the structured interview protocol. 

REL Southeast collaborated with the Just Read, Llorida! staff on identifying which schools 
to interview. REL Southeast anonymized the list of 300 lowest performing schools and then 
randomly ordered the schools by district-school identifier in Excel. That is, in the cell next to 
each district-school identifier, a random number was created using the “=rand()” command. 
The list was then sorted by the random number; this process was repeated 10 times. The 
district and school names were then re-linked to the district-school identifier, and the 
randomly-sorted list of schools was given to Just Read, Florida! staff. REL Southeast suggested 
conducting as many interviews as possible by going down the randomly sorted list of schools. 

The Just Read, Florida! office of the Florida Department of Education conducted nine 
interviews — one per school — using the structured interview protocol developed by REL 
Southeast. 4 Of those nine interviews, six were with staff from one of the lowest perform- 
ing schools in 2014/15 (67 percent of interviews) and three were with staff from one of 
the lowest performing schools in 2014/15 and one prior year (33 percent of interviews); 
no interviews were conducted with staff from one of the lowest performing schools in 
2012/13-2014/15. Interviews were conducted with principals, assistant principals, reading 
coaches and other staff responsible for implementing the extra hour of instruction. In 
some cases the interview was conducted with only the principal (six of nine schools) or 
assistant principal (two of nine schools) at each school; in other cases the reading coach 
(five of nine schools), counselor (one of nine schools), or curriculum resource teacher (one 
of nine schools) joined the interview. 

The Just Read, Florida! staff conducted the interviews in pairs, but each interviewer com- 
pleted his or her own interview protocol. The Florida Department of Education provided 
the study team with scanned copies of the completed interview protocols. The study team 
double-blind entered the information from the protocols into a database for analysis and 
created a single file for each pair of interviews. 

Methodology 

Calculating school reading performance. The Florida Department of Education’s 
(2014c) list of the 300 lowest performing schools in 2014/15 included the school reading 
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performance variable used to identify the lowest performing schools. Because this variable 
was not available for other elementary schools, the study team had to compute school 
reading performance in 2013/14 for those schools. School reading performance is the sum 
of the percentage of students at each school scoring at or above achievement level 3 (satis- 
factory) on the reading component of the prior year’s Florida Comprehensive Assessment 
Test 2.0 (FCAT 2.0) and the percentage of students making learning gains from the prior 
year on the reading component of the FCAT 2.0. School reading performance has a theo- 
retical range of 0 to 200; however the observed range in this study is 54-185. 

Descriptive analysis. The study team calculated descriptive statistics (means, standard 
deviations, and cross-tabulations) to describe the location, demographic characteristics, 
and school reading performance of the lowest performing schools and other elementary 
schools. There were no missing data for any of these variables. 

Similar descriptive statistics were calculated to summarize survey findings on how the 
lowest performing schools implemented the extended school day policy. Respondents com- 
pleted 245 surveys (representing 81 percent of the lowest performing schools) though not 
every question was answered on each survey (see tables C2 and C3 in appendix C for 
response rates to each question). Of the 245 responding schools, 142 (58 percent) were 
lowest performing schools in 2014/15, 81 (33 percent) were lowest performing schools in 
2014/15 and one prior year, and 22 (9 percent) were lowest performing schools in 2012/13— 
2014/15. The survey response rate was 100 percent among the lowest performing schools in 
2012/13-2014/15, 88 percent among the lowest performing schools in 2014/15 and one prior 
year, and 75 percent among the lowest performing schools in 2014/15 (table Al). Because 
there was a clear discrepancy in the response rates between cohorts of lowest performing 
schools, missing data were not imputed. 

Qualitative analysis. The study team used qualitative methods and employed a three- 
phase approach to code and analyze interview data collected from district- and school-level 
leaders in each selected district (Coburn, 2004; Osborne-Lampkin & Cohen-Vogel, 2014; 
Osborne-Lampkin, Folsom, & Flerrington, 2014). The same methods were used to code 
and analyze the supplemental survey data and participants’ elaborated responses to survey 
questions. This approach, which focuses on counting concepts that emerge in the data, 


Table Al. Survey response rates, 2015 

Variable 

All lowest 
performing 
schools 

Lowest 
performing 
schools in 
2014/15 

Lowest 
performing 
schools in 
2014/15 and 
one prior year 

Lowest 
performing 
schools in 
2012/13- 
2014/15 

Lowest performing schools 

Number 

304 

189 

93 

22 

Percent 

100 

62 

31 

7 

Returned surveys 

Number 

245 

142 

82 

22 

Percent of surveys from cohort 

81 

75 

88 

100 

Percent of all surveys 

100 

58 

33 

9 


Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day policy compliance survey (see appendix B). 
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enables researchers to explore and build an in-depth understanding of relevant concepts of 
a particular population or phenomenon (Cohen-Vogel, 2011). 

The coding and analysis were also iterative, requiring the study team to think about the 
data, the connections that were made, and the meanings and conclusions being applied 
throughout the process. First, the study team developed and used two separate a priori 
coding frameworks (that is, coding frameworks that included pre-identified themes, codes, 
and categories rather than frameworks that developed codes through inductive analysis 
based on consideration of the components of the interview and survey protocols). For 
example, the supplemental survey data were analyzed against the corresponding survey 
components to gain additional insight into how participants reportedly incorporated the 
extra hour of instruction. 

Second, the study team used a subset of the data to establish inter-rater reliability and 
to refine the coding framework. 5 Specifically, two study team members pattern-coded a 
purposively selected subset of approximately 20 percent of the data files (two of nine inter- 
views; 48 of 238 survey responses with notes). 6 During this second phase, additional emer- 
gent codes were included in the framework for further analysis. 

Third, the study team coded and analyzed the data using the refined coding framework 
and qualitative analysis software. The study team employed strategies to test and confirm 
findings and checked for representativeness in the data. Additional strategies were also 
employed to increase the validity and trustworthiness of the data and findings, including 
outlining steps and decisions made throughout the process. Multiple members of the study 
team performed data analysis and interpretation in order to minimize researcher bias and 
ensure the quality of the conclusions (for example, weighting the evidence, maintaining a 
running log of data for quality and decisions made; Miles & Fluberman, 1994). The study 
team also made efforts to examine counterevidence (Miles, Fluberman, & Saldana, 2014). 
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Appendix B. Florida Department of Education 
compliance survey and structured interview protocol 


This appendix provides the two instruments used to collect implementation information 
from the 300 lowest performing schools. 

Compliance survey 

This section provides the compliance survey distributed to districts that had one of the 300 
lowest performing elementary schools in 2014/15. The Florida Department of Education, 
through the Just Read, Florida! office, distributed this compliance survey electronically 
along with directions for completion and return. This survey was distributed, completed, 
and returned in early 2015. 
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District’s Extended Day Reading Instruction Plans 


School Information 
School Name: 


School Number: 


District Information 
District Name: 


District Number: 


District Contact Name: 


Contact Phone Number: 


Contact Email: 


Please use the checklist to rate the plan's compliance with the numbered criteria according to this scale: 0 — does not meet criterion, 
criterion, 3 — exceeds criterion. Enter comments only if needed to clarify rating. 

1 — partially meets criterion, 2 — meets 

1. Student Enrollment 

Extended Day Reading Instruction Plans 

A plan has been implemented that: 

Alignment of Statutory Requirements 

0 

1 

2 

3 


District Review Comments 

1. Provided parents notification that all students 
will have an added hour of reading instruction 
each day. (evidence attached) 

For the 2012-2013 and 2013-2014 fiscal years, 

in each school district that has one or more of 

the 100 lowest performing elementary schools 
based on the state reading assessment, priority 
shall be given to providing an additional hour 
per day of intensive reading instruction beyond 
the normal school day for each day of the entire 
school year for the students in each school. 






Extended Day Reading Instruction Plans 

A plan has been implemented that: 

Alignment of Statutory Requirements 

0 

1 

2 

3 


District Review Comments 

2. Provided flexibility so that eligible students 
that scored Florida Comprehensive Assessment 
Test Reading Level 5 may choose to participate 
in one hour of extended reading instruction each 
day, but are not required to do so. (evidence 
attached) 

Students enrolled in these schools who have 

level 5 assessment scores may participate in 
the additional hour of instruction on an optional 
basis. 








B-3 


District’s Extended Day Reading Instruction Plans 


II. Teachers 

The district plan includes selection of highly 
qualified teachers who have: 

Alignment of Statutory Requirements 

0 

1 

2 

3 

District Review Comments 

3. Evidence of prior success teaching reading 
to struggling readers as indicated by various 
assessment data and student work samples over 
time. Teachers should be highly effective K-5 
teachers who are Reading Certified or Endorsed. 

This additional hour of instruction must be 

provided only by teachers or reading specialists 
who are effective in teaching reading. 






III. Reading Program/Materials 

The design of the local reading program and the 
plan for reading instruction/intervention includes: 

Alignment of Statutory Requirements 

0 

1 

2 

3 

District Review Comments 

4. A research-based sequence of intensive 
reading instruction, including instructional 
routines that have been proven to accelerate 
progress of students exhibiting a reading 
deficiency and the integration of social studies, 
science, and mathematics-text reading, text 
discussion, and writing in response to reading. 

The intensive reading instruction delivered in 

this additional hour and for other students shall 

include: research-based reading instruction 
that has been proven to accelerate progress 
of students exhibiting a reading deficiency; 
the integration of social studies, science, and 
mathematics-text reading, text discussion, and 
writing in response to reading. 






The design of the local reading program and the 
plan for reading instruction/intervention includes: 

Alignment of Statutory Requirements 

0 

1 

2 

3 

District Review Comments 

5. One or more of the following types of 
research-based curricula designed to support 
and accelerate student development in the 
components of learning to reading: oral language, 
including vocabulary: phonological awareness 
and phonics; and text reading that support 
accuracy, fluency, and comprehension with more 
extensive opportunities for guided practice, error 
correction, and feedback: 

■"Comprehensive intervention program; 

Targeted intervention program; 

"Supplemental reading program, including 
technology. 

The intensive reading instruction delivered in 

this additional hour and for other students 

shall include explicit and systematic reading 
development in phonemic awareness, phonics, 
fluency, vocabulary, and comprehension, with 
more extensive opportunities for guided practice, 
error correction, and feedback. 
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District’s Extended Day Reading Instruction Plans 


IV. Assessment 

The plan for assessing students includes: 

Alignment of Statutory Requirements 

0 

1 

2 

3 

District Review Comments 

6. Use of assessment and progress monitoring 
results that identify the overall learning needs 
of each student in order to distinguish whether 
their instructional priority is decoding (phonemic 
awareness, phonics, fluency) or text meaning 
(comprehension, vocabulary). This data should 
be used to initially place the student and plan for 
instruction. 

The intensive reading instruction delivered in 
this additional hour and for other students shall 

include: differentiated instruction based on 

student assessment data to meet students' 

specific reading needs. 






7. Regular monitoring (may be informal) of 
students' progress and subsequent adjustment 
of instruction, as needed. 

The intensive reading instruction delivered in 
this additional hour and for other students shall 

include: differentiated instruction based on 

student assessment data to meet students’ 

specific reading needs. 






V. Instruction 

The district must create a reading schedule that 
facilitates interactive, challenging instruction. 

Alignment of Statutory Requirements 

0 

1 

2 

3 

District Review Comments 

8. Instruction must be differentiated and include: 

small group instruction based on student 
assessment data to meet students' specific 
reading needs and include formative assessment 
with writing, and extensive reading from a wide 
variety of texts, all verified with data logs. 

The intensive reading instruction delivered in 

this additional hour and for other students shall 

include: differentiated instruction based on 

student assessment data to meet students’ 

specific reading needs. 






VI. Staff Details 

9. How are you staffing the extra hour of reading instruction? 





District Review 

Comments 


A. Select all that apply The student's regular classroom teachers are providing the instruction 



Other staff are used for the extra hour of reading instruction 


B. Please check who Staff used to 

the other staff are provide extra Last Additional District Review 

for the extra hour of Staff Type hour instruction year this year Total Comments 

reading instruction Reading Coaches 

and indicate how many 







serve your school. Teachers 






number of staff for Paraprofessionals 






each category, year ... 

Volunteers 

and total 






Other 







10. If students' regular teachers do not provide the extra hour of instruction, what are the 
teachers doing during the extra hour? 



District’s Extended Day Reading Instruction Plans 


VII. Instructional Time 

11. How did your school add the required extra hour of 
reading instruction to the school day? (Select all that 
apply) 

Extended the 

bell schedule at 

the beginning of 
the day 

Extended the 

bell schedule at 
the end of day 

Rearranged the 

instructional 

day (provided 

additional 

instruction at a 

different time of 
day) 

Other (please explain) 

District Review 

Comments 

Please select all that apply by placing an X or comment 
in the place below the strategies applied at your school 






12. How many minutes of reading instruction are 
provided during the following blocks per day for an 
average student in your school? 

Standard 

reading block — 
this does not 

include the extra 

hour (e.g., Tier 1 
instruction) 

Intervention 

block (e.g., Tier 

2 instruction) 

Other reading 

related block 

including 

additional hour 

Total Reading 
instruction per 
day for 2013-14 

Total Reading 
instruction per 
day for 2012-13 

District Review 

Comments 

Please report for all categories that apply 








VIII. Student Grouping 


13. Are students grouped differently during the extra hour than during the regular instructional block? 


Check all Approximate 

that apply group size District Review Comments 

extra hour of reading whole class/large group 
instruction? 


14. How are students 

grouped during the Grouping 


small group/individual 


students are grouped homogeneously according to ability/needs 


students are grouped heterogeneously 



Structured interview protocol 


This section provides the structured interview protocol used by the Florida Department of 
Education, through the Just Read, Florida! office, to interview a stratified random sample 
of schools that implemented the extended school day policy. Regional Educational Labora- 
tory Southeast developed this interview protocol and provided substantial training to the 
interviewers at the request of the Florida Department of Education. 
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FACTORS FOR GAINS IN STUDENT ACHIEVEMENT THIS YEAR 


Say: First, I am going to ask you about factors you believe have, generally, impacted gains in student achievement in your school this year. 


Questions and possible responses 


Other/Notes 


Ask: Describe what you think are the main factors for gains in student achievement in your school this year. 


[Interviewer: listen for and mark] 


□ Negative response 

□ We’re still struggling/No gains or drop 

□ Too early to tell 

□ Additional hour effect 

□ Structure 

□ Schedule 

□ Providing additional planning time for teachers 

□ Curricular changes or approach to teaching 

□ Using data to inform instruction 

□ Differentiating instruction 

□ Coaching 

□ Grouping 

□ Staffing and leadership changes 

□ New leadership 

□ Reconstitute faculty 

□ Added coaches or reading leaders 

□ Professional Development/Training 

□ Specific training 

□ Cooperative learning 

□ Ongoing PD or Coaching 

□ Other outside involvement 

□ Parental 

□ Other outside involvement 

□ Other (explain) 
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STRUCTURE & TIME 


Say: Now, 1 am going to ask you about how your school provided the required extra hour of reading instruction in the school day. 

Questions and possible responses 

Other/Notes 

Ask: Describe how your school provided the required extra hour of reading instruction in the school day. 

[Interviewer: listen for and mark, and if necessary, 

Probe: “To what degree did this process include extending the school day?" 

Probe: “To what degree did you rearrange the instructional day?” 

Probe: “Can you describe other ways that you provided the extra hour?”] 

□ Added the hour by extending the school day 

□ School started earlier 

□ School ended later 

□ Rearranged the instructional day 

□ Varied (explain) 

□ Other (explain) 


Ask: How many minutes of reading instruction are provided during the following blocks per day for an average student in your school? 

[Interviewer: Probe for each specific block] 

Block 

Minutes 


Standard reading block (e.g., Tier 1) 


Intervention block (e.g., Tier 2 or 3) 


Mandated extra hour compared to last year’s 

schedule 


Other reading related block (explain) 




B-10 


STAFFING 


Say: Now, 1 am going to ask you about staffing, including any staffing changes. 1 will also ask about shifts in the student body. 

Question and possible responses 

Other/Notes 

Ask: How long have you been the principal of this school? 


[Interviewer: If you are not talking to the principal, ask “How long has the current principal been at your school?] 


Principal has been here for years 


Ask: Describe any major staffing changes that occurred between the 2013/14 and 2014/15 school years. 

[Interviewer: listen for and mark, and if necessary, 

Probe: “When was the most recent leadership change?" 

Probe: “Tell me about any major restructuring of faculty/staff. For example, has your school recently been reconstituted?”] 


□ 

New leadership in 

school year 

□ 

Reconstitute faculty/staff in 

school year 

□ 

Added staff 


□ 

No major changes 


□ 

Other (explain) 



Ask: Describe who is responsible for providing the reading instruction in the extra hour. 

[Interviewer probe for (each): “To what degree did your school use (reading coaches) to provide the extra hour of reading instruction?"] 

□ The students’ regular teachers are providing the instruction 

□ Reading coaches 

□ Reading interventionists 

□ Teachers from another school 

□ Paraprofessionals 

□ Volunteers 

□ Tutors 

□ Other (explain) 
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Question and possible responses 

Other/Notes 

Ask: Describe how you identified who teaches the extra hour of instruction. 

[Interviewer: listen for and mark, and if necessary, probe: "What was your metric of determining eligibility to teach the extra hour?”] 

□ Teachers who had the state reading endorsement 

□ Teachers with higher records of performance according to assessment data 

□ Teachers with more experience 

□ Teachers with higher levels of education 

□ Teachers with more professional development 

□ Younger/new/more recent teachers 

□ Recommendations by administrative staff 

□ Teachers volunteered to teach the extra hour 

□ District or teacher unions decides 

□ Other (explain) 


Ask: Describe the makeup of your instructional staff/coaches (including paraprofessionals, volunteers, and tutors) responsible for teaching the extra hour of reading this year 
compared to last year. 

[Interviewer: Probe for each staff type for last year and this year using this probe: “How many (reading coaches) did you have last year and how many do you have this year?”] 

Staff Type 

# last year 

# this year 



Reading coaches 



Reading interventionists 



Teachers from another 

school 



Paraprofessionals 



Volunteers 



Tutors 




Other (explain) 
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Question and possible responses 

Other/Notes 

Ask: If students' regular teachers do not provide the extra hour of instruction, describe what the regular teachers are responsible for doing during the extra hour. 

[Interviewer: listen for and mark] 

□ N/A - same teachers 

□ Coaching in another classroom 

□ Acting as an aide or co-teacher to the reading instructor for the extra hour 

□ Non-reading instruction 

□ Receiving professional development or coaching 

□ Working with student data 

□ Planning time 

□ Non-instructional duties (lunch duty, hall monitoring) 

□ Other (explain) 


Ask: Describe any major student body changes over the last 5 years. 

[Interviewer: listen for and mark] 

□ N/A - none 

□ This is a new school 

□ General rezoning in the district 

□ New school opened - students left here 

□ Old school closed - students shifted here 

□ General economic decline 

□ Natural disasters (hurricane/sink holes) 

□ Other (explain) 
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Question and possible responses 

Other/Notes 

Ask: Describe any professional development or other preparation offered to the teachers who provide the extra hour of reading instruction. 

[Interviewer: listen for and mark, and if necessary, 

Probe: “Could you describe the specific professional development curriculum or model? 

Probe: “What was/is the frequency of the PD?"] 

□ N/A - none provided 

□ Specific training provided as a one-time PD 

Name: 

□ Ongoing PD 

Name: 

□ Professional learning community (ongoing) 

□ Coaching support (ongoing) 

□ Cooperative learning 

□ Other (explain) 


CURRICULUM & INSTRUCTION 

Say: Now, 1 am going to ask you about the curriculum you use for reading instruction and how instruction is structured. 

Ask: Name or describe the curriculum selected/used by your school for each of the following blocks of instruction. 

[Interviewer: Probe for each specific (instructional block) using this probe: “What was the name of the curriculum used in the (standard reading)?"] 

Instructional Block 

Curriculum Name 


Standard reading block using the core 
curriculum (Tier 1) 


Intervention block 

(Tier 2 or 3 instruction) 


The required additional hour of 

instruction 


Other reading related block (explain) 


Question and possible responses 

Other/Notes 

Ask: On scale of 1 (not at all appropriate) to 6 (very appropriate), how appropriate do you feel the curriculum(s) were for your students/school? What changes would you make? 

[Interviewer: listen for and mark] 

1 2 3 4 5 6 

□ New/updated curriculum 

□ Different curriculum 

□ Different publisher 

□ Other (explain) 
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Ask: Describe how students are grouped for instruction in the regular reading instructional block. 

[Interviewer: 

Probe: “To what extent are students homogeneously grouped?" 

Probe: "To what extent are students heterogeneously grouped?” 

Probe: “Can you describe other ways that students were grouped?” 

Probe for each specific (grouping method) using this probe: “What is the approximate group size in (small groups)?”] 

□ Homogeneously according to ability/needs 

□ Heterogeneously 

□ Varies 

□ Other (explain) 


Grouping Method 

Approximate Group Size 


Whole class/large group 


Small group 


Pair or One-on-one 


Question and possible responses 

Other/Notes 

Ask: Describe the criteria used when deciding how to group students for regular reading instructional. 

[Interviewer: Probe for each specific (decision): “To what extent were (teacher recommendations) considered?”] 

□ None 

□ Teacher recommendation 

□ Grades 

□ Progress monitoring data 

□ FCAT data 

□ Random assignment 

□ Other (explain) 
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Ask: Describe how students are grouped for the extra hour of instruction. 

[Interviewer: 

Probe: “To what extent are students homogeneously grouped?” 

Probe: “To what extent are students heterogeneously grouped?” 

Probe: “Can you describe other ways that students were grouped?” 

Probe for each specific (grouping method) using this probe: “What is the approximate group size in (small groups)?”] 

□ No different/Same grouping 

□ Homogeneously according to ability/needs 

□ Heterogeneously 

□ Varies 

□ Other (explain) 


Grouping Method 

Approximate Group Size 


Whole class/large group 


Small group 


Pair or One-on-one 


Question and possible responses 

Other/Notes 

Ask: Describe the criteria used when deciding how to group students for the extra hour of instruction. 

[Interviewer: Probe for each specific (decision): "To what extent were (teacher recommendations) considered?”] 

□ None 

□ Teacher recommendation 

□ Grades 

□ Progress monitoring data 

□ FCAT data 

□ Random assignment 

□ Other (explain) 
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OVERALL OPINION ABOUT THE EXTRA HOUR REQUIREMENT 


Say: Lastly, I would like to ask your overall opinion of the extra hour requirement. 

Ask: On scale of 1 (not at all prepared) to 6 (very prepared) how prepared do you feel your school was to implement the additional hour? What changes would you make? 


[Interviewer: listen for and mark, and if necessary, 

Probe: "What would you do differently in your school if you had the chance to do this over?”] 


1 2 3 4 5 6 

□ Earlier notice 

□ More funding 

□ More guidance from the district 

□ More guidance from the state 

□ Other (explain) 


Question and possible responses 

Other/Notes 

Ask: On scale of 1 (not at all prepared) to 6 (very prepared) how prepared do you feel your district was to support your school in the implementation of the additional hour? 

Ask: What changes would you make? 

[Interviewer: listen for and mark, and if necessary, 

Probe: “What support would you request from your district if you had the chance to do this over?’’] 

1 2 3 4 5 6 

□ Earlier notice 

□ More funding 

□ More district provided PD 

□ Other (explain) 


Ask: On scale of 1 (not at all enthusiastic) to 6 (complete enthusiastic) how enthusiastic were your staff about implementing the additional hour of instruction? 

[Interviewer: listen for and mark] 

1 2 3 4 5 6 
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Ask: On scale of 1 (not at all) to 6 (very much) how much did teacher union involvement influence implementation? 

Ask: How? 

[Interviewer: listen for and mark, and if necessary, Probe: How would you describe the union's involvement and influence?"] 

1 2 3 4 5 6 

□ Influenced who could/could not provide the extra hour 

□ Determined who could/could not provide the extra hour 

□ Other (explain) 


Question and possible responses 

Other/Notes 

Ask: Describe what you see as benefits of the extra hour policy. Describe any barriers you see in implementing the extra hour. 

[Interviewer: listen for and mark, and if necessary, 

Probe for (benefits/barriers) What would you describe as (benefits/barriers) from/to the extra hour of instruction?] 

Benefits 

□ We have our students longer, more time for instruction 

□ Opportunity to provide PD to teachers 

□ Opportunity to provide more planning for teachers 

□ Other (explain) 

Barriers 

□ Funding 

□ Not enough time to plan 

□ Other (explain) 


Ask: What are your plans next year regarding this additional hour of reading instruction? 

[Interviewer: listen for and mark] 

□ Will only do if required 

□ Will continue regardless if required 

□ Will not continue if not required 

□ Undecided 

□ Other (explain) 


Say: “This concludes our interview.” 

Ask: "Before we close, do you have anything you would like to add or clarify?" 

Say: “On behalf of the Florida Department of Education, Just Read, Florida! 1 thank you for your time and thorough responses. This information will be very valuable and used by 
the Florida DOE to better understand how the extra hour policy is being received and implemented. Best wishes for a successful remainder of the year. If you have any questions 
later or anything you would like to add later, feel free to contact me. 
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POST-INTERACTION FORM 


Complete this last section immediately AFTER the interview is complete. These questions are NOT to be asked of the interviewee, and are to be completed only by the interviewer. 

Question 

Notes 

Did the interviewee ever mention the following without prompting by you? 



□ Research-based instruction 

□ Differentiating instruction 

□ Integrating phonemic awareness, phonics, fluency, vocabulary, and comprehension 

□ Incorporated guided practice, error correction, and feedback 

□ Integrated social studies, science, and mathematics-text reading; text discussion; 
and writing in response to reading 


What was your overall impression of the interviewee’s attitude towards the requirement for an extra hour of instruction in their school? 

□ Positive 

□ Neutral 

□ Negative 


Additional notes or comments 



Appendix C. Supplemental tables of school characteristics, 
school reading performance, and survey responses 


This appendix provides descriptive statistics for all variables included in this report. Table 
Cl provides descriptive statistics of school characteristics and school reading performance. 
Table C2 provides aggregated and disaggregated descriptive statistics of minutes of instruo 
tion provided as reported in the compliance survey. Table C3 provides aggregated and 
disaggregated descriptive statistics from the compliance survey. 


Table Cl. Descriptive statistics of school characteristics in 2014/15 and school reading performance 
in 2013/14 and 2014/15 


Variable 

Full sample 

Other elementary 
schools 

Lowest performing 
schools in 
2014/15 

Lowest performing 
schools in 2014/15 
and one prior year 

Lowest performing 
schools in 
2012/13-2014/15 

Number of schools 


Number 


1,798 


1,494 


189 


93 


22 


Percent 


100 


83 


11 


Enrollment in 2014/15 


Mean 


640.1 


660.0 


566.0 


510.1 


474.2 


Standard deviation 


227.6 


228.3 


210.4 


176.2 


119.7 


Range 


31-1,929 


50-1,929 


31-1,191 


135-1,023 


252-660 


Title I schools 


Number 


1,180 


880 


185 


93 


22 


Percent 


66 


59 


98 


100 


100 


Percentage of racial/ethnic minority students in the school 


Mean 


60.1 


56.1 


76.7 


84.0 


84.2 


Standard deviation 


27.5 


26.5 


22.5 


24.7 


24.8 


Range 


3-99 


3-99 


10-99 


10-99 


10-99 


Percentage of students eligible for the federal school lunch program 


Mean 


62.2 


59.4 


73.5 


79.5 


79.0 


Standard deviation 


24.9 


23.9 


24.4 


26.5 


25.2 


Range 


0-99 


0-99 


8-99 


10-99 


10-99 


Percentage of students at the school reading at or above achievement level 3 (satisfactory) on the FCAT 2.0 in 2013/14 


Mean 


58.7 


63.4 


38.0 


32.8 


25.8 


Standard deviation 


16.1 


13.2 


5.7 


5.4 


7.6 


Range 


13-99 


28-99 


18-54 


18-44 


13-40 


Percentage of students at the school making learning gains from the prior year on the reading component of the FCAT 2.0 in 2013/14 


Mean 


68.2 


70.5 


56.2 


57.9 


55.4 


Standard deviation 


8.8 


7.1 


6.1 


7.4 


8.6 


Range 


28-96 


48-96 


28-73 


38-72 


41-75 


School reading performance in 2013/14 


Mean 


126.9 


133.9 


94.6 


90.7 


81.2 


Standard deviation 


22.9 


18.0 


7.5 


9.3 


14.0 


Range 


54-185 


104-185 


63-103 


65-103 


54-102 


English language arts achievement test scores in 2014/15 


Mean 


52.4 


56.5 


35.1 


28.5 


21.2 


Standard deviation 


16.4 


14.4 


8.4 


6.7 


5.8 


Range 


5-96 


5-96 


13-83 


15-47 


10-31 


FCAT 2.0 is Florida Comprehensive Assessment Test 2.0. 

Source: Authors' analysis of data from Florida Department of Education (2014a, 2015, 2016b, 2016c) of data obtained by special 
request from the Florida Department of Education extended school day policy compliance survey (see appendix B). 
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Table C2. Descriptive statistics of amount of instruction provided, in 2013/14 and 
2014/15 




Lowest performing 

Lowest performing 


Lowest performing 

schools in 2014/15 

schools in 2012/13- 

Variable 

schools in 2014/15 

and one prior year 

2014/15 


Minutes of reading instruction provided in 2013/14 


Number of schools reporting 125 81 20 

Mean 127.8 160.7 183.0 

Standard deviation 29.2 36.1 34.3 

Range 90-210 90-240 90-225 

Minutes of reading instruction provided in 2014/15 

Number of schools reporting 135 81 22 

Mean 179.9 182.4 184.3 

Standard deviation 27.4 22.2 12.9 

Range 60-255 150-240 150-210 

Change in minutes of reading instruction provided between 2013/14 and 2014/15 


Number of schools reporting 123 

Mean 53.9 

Standard deviation 23.9 

Range 60-130 


81 

21.7 

30.0 

30-100 


20 

1.5 

28.3 

30-90 


Source: Authors' analysis of data obtained by special request from the Florida Department of Education ex- 
tended school day policy compliance survey (see appendix B). 


Table C3. Frequencies of responses from the compliance survey, 2015 





Lowest 

Lowest 




Lowest 

performing 

performing 



performing 

schools in 

schools in 



schools in 

2014/15 and 

2012/13 




2014/15 

one prior year 

2014/15 

Question 

Response 

Number 

Percent 

Number 

Percent 

Number Percent 

1. Provided parents notification that all 

Does not meet criterion 

3 

2 

0 

0 

1 

5 

students will have an added hour of reading 

Partially meets criterion 

3 

2 

2 

2 

2 

9 

instruction each day. (n = 245) 

Meets criterion 

87 

61 

39 

48 

16 

73 


Exceeds criterion 

49 

35 

40 

49 

3 

14 

2. Provided flexibility so that eligible students 

Does not meet criterion 

7 

5 

5 

6 

2 

9 

that scored Florida Comprehensive Assessment 

Partially meets criterion 

4 

3 

2 

3 

1 

5 

Test Reading Level 5 may choose to participate 
in one hour of extended reading instruction each 

Meets criterion 

87 

63 

42 

54 

17 

77 

Exceeds criterion 

40 

29 

29 

37 

2 

9 

day, but are not required to do so. ( n = 238) 

3. Evidence of prior success teaching reading 

Partially meets criterion 

27 

19 

8 

10 

6 

29 

to struggling readers as indicated by various 

Meets criterion 

82 

59 

45 

56 

10 

48 

assessment data and student work samples 

Exceeds criterion 

31 

22 

28 

35 

5 

24 


overtime. Teachers should be highly effective 
K-5 teachers who are Reading Certified or 
Endorsed, (n = 242) 


4. A research-based sequence of intensive 

Partially meets criterion 

5 

4 

1 

1 

0 0 

reading instruction, including instructional routines 

Meets criterion 

100 

71 

50 

62 

17 77 

that have been proven to accelerate progress 
of students exhibiting a reading deficiency and 
the integration of social studies, science, and 
mathematics-text reading, text discussion, and 
writing in response to reading, (n = 244) 

Exceeds criterion 

36 

26 

30 

37 

5 23 

(continued) 
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Table C3. Frequencies of responses from the compliance survey, 2015 (continued) 



Lowest 
performing 
schools in 
2014/15 

Lowest 
performing 
schools in 
2014/15 and 
one prior year 

Lowest 
performing 
schools in 
2012/13- 
2014/15 

Question 

Response 

Number 

Percent 

Number Percent 

Number 

Percent 

5. One or more of the following types of 

Partially meets criterion 

3 

2 

0 

0 

0 

0 

research-based curricula design to support 

Meets criterion 

100 

72 

46 

57 

18 

82 

and accelerate student development in the 
components of learning to read: oral language, 
including vocabulary; phonological awareness 
and phonics; and text reading that supports 
accuracy, fluency, and comprehension with 
more extensive opportunities for guided 
practice, error correction, and feedback: 
♦Comprehensive intervention program 
♦Targeted intervention program 
♦Supplemental reading program, including 
technology, (n = 242) 

Exceeds criterion 

36 

26 

35 

43 

4 

18 

6. Use of assessment and progress monitoring 

Partially meets criterion 

3 

2 

0 

0 

0 

0 

results that identify the overall learning 

Meets criterion 

100 

75 

63 

79 

19 

86 

needs of each student in order to distinguish 
whether their instructional priority is decoding 
(phonemic awareness, phonics, fluency) or 
text meaning (comprehension, vocabulary). 

This data should be used to initially place the 
student and plan for instruction, (n = 235) 

Exceeds criterion 

30 

23 

17 

21 

3 

14 

7. Regular monitoring (may be informal) of 

Partially meets criterion 

4 

3 

0 

0 

0 

0 

students’ progress and subsequent adjustment 

Meets criterion 

103 

77 

60 

76 

19 

86 

of instruction, as needed, (n = 235) 

Exceeds criterion 

27 

20 

19 

24 

3 

14 

8. Instruction must be differentiated and 

Does not meet criterion 

1 

1 

0 

0 

0 

0 

include: small group instruction based on 

Partially meets criterion 

9 

6 

4 

5 

2 

9 

student assessment data to meet students' 

Meets criterion 

100 

71 

57 

72 

17 

77 

specific reading needs and include formative 
assessment with writing, and extensive reading 
from a wide variety of texts, all verified with 
data logs, (n = 241) 

Exceeds criterion 

30 

21 

18 

23 

3 

14 

VI. Staff details 

9a. How are you staffing the extra hour of 
reading instruction? (n = 221) 

Other staff used for the 
extra hour of reading 

instruction 

20 

16 

12 

17 

3 

14 


The students’ regular 

classroom teachers are 

providing the extra hour 
of reading instruction 

2 

2 

0 

0 

0 

0 


Both 

106 

83 

60 

83 

18 

86 

9b. Staffing changes: Reading coaches 

Less 

1 

1 

8 

12 

1 

5 

(n = 178) 

More 

60 

64 

21 

32 

2 

11 


No change 

33 

35 

36 

55 

16 

12 

9b. Staffing changes: Teachers 

Less 

3 

3 

11 

16 

6 

29 

(n = 191) 

More 

81 

81 

40 

57 

9 

43 


No change 

16 

16 

19 

27 

6 

29 

9b. Staffing changes: Paraprofessionals 

Less 

2 

2 

1 

2 

0 

0 

(n = 156) 

More 

54 

64 

21 

38 

8 

47 


No change 

28 

33 

33 

60 

9 

53 

9b. Staffing changes: Volunteers 

Less 

0 

0 

2 

5 

1 

13 

(n = 116) 

More 

12 

19 

8 

12 

1 

13 


No change 

51 

81 

35 

83 

6 

75 

(continued) 
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Table C3. Frequencies of responses from the compliance survey, 2015 (continued) 



Lowest 
performing 
schools in 
2014/15 

Lowest 
performing 
schools in 
2014/15 and 
one prior year 

Lowest 
performing 
schools in 
2012/13- 
2014/15 

Question 

Response 

Number 

Percent 

Number Percent 

Number Percent 

9b. Staffing changes: Other staff 

Less 

0 

0 

8 

14 

1 6 

(n = 151) 

More 

50 

64 

20 

35 

5 29 


No change 

27 

35 

29 

51 

11 65 

VII. Instructional time 

11. How did your school add the required extra 
hour of reading instruction to the school day? 

(n = 244) 

Extended the bell 

schedule at the 
beginning of the day 

5 

4 

7 

9 

1 5 


Extended the bell 

schedule at the end of 

the day 

52 

37 

23 

29 

7 32 


Rearranged the 
instructional day 
(provided additional 

instruction at a different 
time of day) 

24 

17 

11 

14 

1 5 


Combination: Extended 

the bell schedule at the 

beginning and end of 
the day. 

18 

13 

9 

11 

3 14 


Combination: Extended 

the bell schedule at 

the end of the day 
and rearranged the 
instructional day 

28 

20 

26 

33 

8 36 


Combination: Extended 

the bell schedule at 

the beginning of the 
day and rearranged the 
instructional day 

7 

5 

1 

1 

0 0 


Combination: Extended 

the bell schedule at the 
beginning and end of the 
day, and rearranged the 
instructional day 

5 

4 

3 

4 

2 9 


Other 

3 

2 

0 

0 

0 0 

VIII. Student grouping 

14. How are students grouped during the extra 
hour of reading instruction? (n = 231) 

Whole class/large group 
only 

2 

2 

2 

3 

0 0 


Small group/individual 
only 

48 

36 

30 

40 

10 46 


Both small group/ 

individual and whole 

class/large group 

83 

62 

44 

58 

12 55 


Homogeneous only 

74 

56 

46 

60 

12 57 


Heterogeneous only 

5 

4 

10 

13 

1 5 


Both homogeneous and 
heterogeneous 

54 

41 

21 

27 

8 38 

Note: Data are for 235 schools. Headers, questions, and question numbers match the survey provided in appendix B. Categories not 
listed for questions 1-8 had zero frequencies. Questions 10, 12, and 13 are omitted because their response options are not conducive 
to table format. Percentages may not sum to 100 because of rounding. 

Source: Authors’ analysis of data obtained by special request from the Florida Department of Education extended school day policy 
compliance survey (see appendix B). 
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Appendix D. Supplemental statistical tests of significance 
for school characteristics and implementation 
of extended school day policy in 2014/15 


This appendix provides results of supplemental statistical tests of significance. Tests are 
listed in the order in which they are discussed in the main text. 

Region 

A chi-square test revealed that the lowest performing elementary schools were not evenly 
distributed across regions (y 2 = 27.98, df = 12, p = .006). Some regions had significantly 
more lowest performing schools than did other regions. 

Enrollment 

An analysis of variance of enrollment revealed a significant test for cohort (F = 25.49, 
df = 3, p < .001). This suggests that the lowest performing schools were smaller than other 
schools. 

Title I status, racial/ethnic minority, and federal school lunch program eligibility 

A chi-square test revealed that a statistically significant larger proportion of lowest per- 
forming schools than of other elementary schools were Title I schools (y 1 = 177.36, df = 3, 
p < .001). Additionally, analysis of variance revealed that the percentage of racial/ethnic 
minority students (F = 69.54, df = 3, p < .001) and the percentage of students eligible for 
the federal school lunch program (F = 39.99, df = 3, p < .001) were higher in the lowest 
performing schools than in other elementary schools. 

School reading performance 

An analysis of variance of school reading performance revealed a test for cohort for both 
2013/14 (F = 517.56, df = 3, p < .001) and 2014/15 (F = 283.80, df = 3, p < .001). This sug- 
gests that school reading performance was significantly different between the cohorts of 
lowest performing schools. Pairwise comparisons with Tukey’s post hoc test revealed the 
following statistically significant differences for school reading performance in 2013/14: 

• All cohorts of lowest performing schools were lower than other elementary schools 
(p < .001 for all comparisons) 

• The lowest performing schools in 2014/15 were higher than the lowest performing 
schools in 2012/13-2014/15 (p = .002) 

Pairwise comparisons with Tukey’s post hoc test revealed the following statistically signifi- 
cant differences for school reading performance in 2014/15: 

• All cohorts of lowest performing schools were lower than other elementary schools 
(p < .001 for all comparisons). 

• The lowest performing schools in 2014/15 were higher than the lowest performing 
schools in 2014/15 and one prior year (p = .001). 

• The lowest performing schools in 2014/15 were higher than the lowest performing 
schools in 2012/13-2014/15 (p < .001). 
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Minutes of reading instruction 


A repeated measures analysis of variance of minutes of reading instruction revealed a 
statistically significant between-subjects effect for cohort (F = 19.61, df = 2, p < .001), a 
statistically significant within-subjects effect for year (F = 118.63, df = 1, p < .001), and 
a statistically significant time by cohort interaction (F = 55.68, df = 2, p < .001). This 
suggests that the amount of reading instruction in 2014/15 was significantly different from 
that in 2013/14, that the cohorts of lowest performing schools provided significantly dif- 
ferent amounts of instruction, and that the change in the amount of instruction provided 
was different between the cohorts of lowest performing schools. 

Chi-square tests revealed no statistically significant differences in how the cohorts of 
lowest performing schools incorporated the extra hour of instruction into the school day 
(question 11 on the survey; y 2 = 16.762, df = 14, f> = .269). 

Staffing 

Chi-square tests revealed no statistically significant differences in how the cohorts of lowest 
performing schools were staffing the extra hour (question 9a on the survey; yf = 1.538, 

df= 4,1) = . 820). 


Chi-square tests revealed statistically significant differences in how the cohorts of lowest 
performing schools made staffing changes (question 9b on the survey) from 2013/14 to 
2014/15 related to reading coaches {y 2 = 54.86, df = 6, p < .001), teachers (y 2 = 50.52, 
df = 6, p < .001), paraprofessionals (y 1 = 22.67, df = 6, p = .001), and other staff (y 2 = 41.74, 
df = 6, p < .001). However, differences related to volunteers were not statistically significant 
( x 2 = 11.32,d/=6,p = .079). 

Grouping 

Chi-square tests revealed no significant differences in the group sizes of instruction provid- 
ed in the extra hour (question 14 on the survey) among the cohorts of lowest performing 
schools (j( 2 = 1.56, df = 4, p = .816). 

Chi-square tests revealed no statistically significant differences in how students were 
grouped for the extra hour (question 14 on the survey) among the cohorts of lowest per- 
forming schools (y 2 = 8.74, df = 4, p = .068). 

Compliance 

Chi-square tests revealed statistically significant differences between the cohorts of lowest 
performing schools regarding compliance with two of the extended school day policy’s 
eight requirements (questions 1-8 on the survey): parent notification (y 2 = 15.14, df = 6, 
p = .019) and use of a research based curricula (y 2 = 10.89, df = 4, p = .028). Differences in 
compliance with the other six requirements were not statistically significant. Chi-square 
tests ranged from 1.97 to 7.98, with four degrees of freedom; p-values were all greater than 
.092. 
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Notes 


1. The change was introduced when House Bill 5101 amended sections 1011.62(4) (f)(2) 
and 1011.62(9)(a) of the Florida Statutes (http://laws.flrules.org/2014/56). 

2. While alliance members were also interested in whether the changes in the law were 
reflected in implementation, the changes came about after the study was designed and 
after the survey and interview protocols were developed. Therefore, no survey or inter- 
view questions related specifically to the changes in the law. 

3. For an explanation of how learning gains are calculated, see Florida Department of 
Education (2014b). 

4. The Just Read, Florida! office did not have staff dedicated specifically to conduct inter- 
views. Thus the number of interviews was limited by the personnel and time available 
amidst all other departmental staff responsibilities. 

5. Inter-rater reliability in qualitative analysis refers to the issue of whether different 
coders will code the data in the same way. 

6. Whether the size of a sample used to assess reliability is appropriate depends on many 
factors, but a common standard is that it should not be less than 10 percent of the full 
sample (Neuendorf, 2002). It is recommended that a pilot test be conducted to assess 
reliability, followed by an assessment of reliability of another representative sample of 
the full sample to be coded. For example, the study team purposively sampled two 
of the nine interview files as a pilot to assess reliability, checking reliability after the 
coding and analyzing the first file. The study team continued the assessment with one 
additional file to reach inter-rater reliability. Had the study team failed to reach reli- 
ability with these initial samples, it would have been required to continue coding addi- 
tional files until reaching reliability. In other words, if researchers were hypothetically 
analyzing the content of 80 documents, they would be required to pilot a minimum of 
eight files. They would follow with coding an additional eight files (10 percent of the 
full sample), adding files as necessary until coders reached reliability. For the survey 
notes, the study team used 49 files to establish reliability. 
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The Regional Educational Laboratory Program produces 7 types of reports 


Making Connections 

Studies of correlational relationships 


Making an Impact 

Studies of cause and effect 


What’s Happening 

Descriptions of policies, programs, implementation status, or data trends 


What’s Known 

Summaries of previous research 



Stated Briefly 

Summaries of research findings for specific audiences 


Tools 

Help for planning, gathering, analyzing, or reporting data or research 




